Skip to content

fix(harness): preserve selected coding agent for template launches - #809

Merged
BYWallace merged 9 commits into
mainfrom
brettwallace/sap-3121-template-harness
Sep 9, 2026
Merged

fix(harness): preserve selected coding agent for template launches#809
BYWallace merged 9 commits into
mainfrom
brettwallace/sap-3121-template-harness

Conversation

@BYWallace

@BYWallace BYWallace commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Problem and motivation

Choosing Codex in Studio could still launch a template with Claude Code. Old conversations could also be lost when archive backfill stopped at 200 records or failed before cleanup.

Summary and scope

Use one App-level harness selection for the composer and template launches. Preferences initialize and persist it; navigation uses the current choice. Registry failures retain the selection, and unavailable defaults are corrected. Archive at most 200 conversations per pass. Keep source events while work remains or archiving fails. Continue at the next six-hour maintenance pass. Track completed writes across passes so archive size cleanup cannot restart the same work.

Bug and fix

flowchart LR
    A["Select Codex<br/>Use a template"] -->|Before| B["Template launch<br/>defaults to Claude"]
    B --> C["Claude Code starts"]
    A -->|After| D["Read App's<br/>selectedHarness"]
    D --> E["Codex starts"]
    F["Archive up to 200 conversations"] -->|Complete| G["Run event retention"]
    F -->|More work or error| H["Keep source events<br/>Continue on the next six-hour pass"]
    classDef bug fill:#fff1f2,stroke:#be123c,color:#881337
    classDef fix fill:#ecfdf5,stroke:#047857,color:#064e3b
    class B,C bug
    class D,E fix
Loading
Changed area What changes Additions
App, composer, API, registry One harness selection; named default; template actions use it 83
Browser tests Template launches, navigation, fallback, and project start 320
Archive + regression tests 200 per pass; preserve source until complete; track progress 160
README + changesets Behavior and release notes 12
Total 575

Related work

SAP-3121. Codex MCP configuration is separate in #810.

Validation

  • Rebased on latest main (ff414ffb). Workspace build, typecheck, and lint passed.
  • All 37 targeted archive/retention tests passed. Covers 200, 201, and 401 conversations, archive eviction between passes, failures, and retry. The limit and eviction regressions failed before their fixes.
  • Full Harness: 3,988 passed, two failed. The session-manager timeout and Mac watcher assertion also failed before this change; their source files are unchanged. All other package suites and all 10 performance tests passed.
  • Prior UI checks: all 23 template tests passed; the native app started the selected Codex and Claude processes. Full template completion remains unverified. UI tests were not repeated for this archive update.

September 4 native demo of both fixes (before this rebase):

agent-studio-codex-fix-demo.mp4
Checklist and release details

Primary change type

  • Bug fix
  • Documentation
  • Feature
  • Tests
  • Dependency update
  • Maintenance or refactor

Tests and documentation

Added template tests and archive history, failure, and retry tests. Updated the Harness README.

Compatibility and release impact

  • Breaking or externally visible changes: templates use the selected harness. Event cleanup waits for archive work and keeps source data on failure. No migration is required.
  • Changesets: added @sapiom/harness patch notes for selection and archive retention.

Security

  • I have not included secrets, credentials, private data, or unsanitized logs.
  • This pull request does not publicly disclose a suspected vulnerability.

AI assistance

  • I did not use AI assistance for this change.
  • I used AI assistance and have described it below.

Codex implemented the change; validation is recorded above.

Checklist

  • I read CONTRIBUTING.md and followed the contribution policy.
  • This pull request addresses one focused problem.
  • I added or updated tests.
  • I ran the relevant build, typecheck, lint, and test commands; known failures are noted above.
  • I updated documentation.
  • I added a Changeset.
  • I can explain and maintain every submitted change, including any AI-assisted work.

@BYWallace
BYWallace marked this pull request as ready for review September 5, 2026 00:13
@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Review — PR #809 (round 1)

1. Auto-selected agent is still dropped outside the composer (correctness)

NewSessionComposer.tsx:163-180 corrects the selection when the stored/default agent is
not installed, but that correction is only local state — saveUiPrefs runs solely in
pickHarness (NewSessionComposer.tsx:201-205). So on a machine where Claude Code is
absent, loadUiPrefs().preferredHarness stays undefined and preferredHarness()
(App.tsx:1749) returns "claude-code".

Failure: fresh install, only Codex installed, composer shows Codex. Open templates from
the rail (or the palette, or a ?template=… deep link — none of which get the composer
override) and use any template → createSessionAt(cwd, "claude-code"), i.e. a session on
an adapter the machine does not have. That is the exact symptom in the PR title, just
reached through the entry points the override doesn't cover.

The test matrix shows the gap: template-harness.spec.ts runs the
__MOCK_UNINSTALLED_HARNESSES__ case for composer / gallery-detail / starter-detail
only, while the rail / palette / deep-link loop uses an explicitly picked (therefore
persisted) Codex. One-line fix: persist the corrected id inside the registry effect, which
also repairs App.tsx:1708, :1959 and :2875.

2. Back/forward navigation discards the composer override (correctness)

applyVisit (App.tsx:1152) calls setTemplatesOpen(visit.kind === "templates") with no
second argument, so replaying a templates visit resets templatesView to
{ harness: undefined } and the launch falls back to preferredHarness(). Harmless when
the pick was persisted — but the override exists precisely for the case where it wasn't,
which this PR tests ("preferences cannot be saved"). Sequence: pick Codex with
setItem throwing → browse templates → exit → Back → use a template → Claude Code.
Either carry the harness on the NavigationVisit record or accept it and drop the claim in
the README that the composer selection survives a failed save.

Nits

  • README.md:24-27 documents an implementation quirk to npm consumers ("even when
    preferences cannot be saved", "Other gallery entry points use the saved preference"). A
    reader can't act on it; either make the behaviour uniform (finding 1) or cut the sentence
    to "templates launch with your selected coding agent".
  • The same hardcode the PR removes is still in the neighbouring clone/start flows —
    App.tsx:2055, :2481, :2537, :3812, :3875 — where a user who picked Codex hits
    the identical surprise. Out of scope here; worth a follow-up ticket rather than scope creep.
  • setTemplatesOpen changed from a useState setter to a useCallback; only the dep array
    at App.tsx:1220 was updated, applyVisit's was not. Both are stable so nothing breaks,
    and eslint src --ext .ts never sees web/src — flagging only so the inconsistency isn't
    taken as intent.

Verdict

Approach is right (derive templatesOpen from a per-visit record; thread the selection
through the call), changeset level is correct for a behaviour fix, and no confidentiality or
published-surface problems: the changeset, README and test fixtures name only Sapiom and the
public adapters. Finding 1 leaves the headline bug reachable on an install without Claude
Code — worth fixing before merge; finding 2 is a small follow-up.

@BYWallace
BYWallace force-pushed the brettwallace/sap-3121-template-harness branch from 0d01cac to bba398d Compare September 5, 2026 00:48
@ratataque

Copy link
Copy Markdown
Contributor

wouldn't it be better to have a global state that tracks the harness that you chose instead of passing it to the template creation ? so that we don't have same issue going forward and have only 1 source of truth ?

@github-actions

github-actions Bot commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Review — PR #809 (round 2)

Delta reviewed: adfee74cbba398d4 (the round-1 marker sha is the merge commit for
adfee74c; the two new commits lift the selection into App).

Both round-1 findings are fixed: the registry correction now lives in App.tsx:259-272 so
every entry point (rail, palette, deep link) sees it, and NavigationVisit carries
harness so Back/Forward replays the gallery visit's selection. The new tests cover both,
including the uninstalled-Claude-Code case for rail/palette/deep-link. README nit and the
applyVisit dep-array nit are addressed too.

1. NEW — a failed registry fetch turns template launches into a no-op

App.tsx:2149: const registry = harnessEntries ?? (await harness.listHarnesses()). The
App-level effect swallows its own failure (.catch(() => {}), :270), so after a failed
probe harnessEntries stays null and every template use re-fetches. If that call rejects,
handleUseTemplate now throws before creating anything, and the composer caller drops it —
void handleUseTemplate(...) at :2267 has no catch. Failure: adapter-probe endpoint
5xx/timeout while sessions still create fine → clicking a template in the composer does
nothing, no toast, no session. Before this push it launched with the selected harness.
Fix: await harness.listHarnesses().catch(() => FALLBACK_HARNESSES) (already imported).

2. Round-1 nit 2 — still two sources of truth inside one component

preferredHarness() (App.tsx:1788) still reads persisted prefs, and the auto-correction
is never persisted, so :1747, :1998, :2922 and :386 keep booting claude-code on a
machine without it while the composer displays Codex. Out of scope in round 1; now that
selectedHarness is in the same scope it is a one-argument change, and it is what the
human reviewer asked for.

Nothing else new; no confidentiality or published-surface problems in the delta (prose
names only Sapiom and public adapters, __MOCK_UNINSTALLED_HARNESSES__ follows the
existing __MOCK_* idiom in api.ts, changeset unchanged and still correctly a patch).

Verdict: finding 1 is worth a one-line guard before merge; the rest of the push
resolves round 1.

@BYWallace

BYWallace commented Sep 5, 2026

Copy link
Copy Markdown
Contributor Author

Updated in f608e54: selectedHarness is now the single App-level source of truth. I removed the template/navigation snapshots and the remaining launch-time preference reads. Back/Forward uses the current choice, including when saving fails. The registry-fetch failure is handled too; all 71 focused browser tests pass.

On the earlier dependency-array nit, the flagged array already belonged to applyVisit and included setTemplatesOpen. The neighboring explicit Claude clone/start choices predate this template fix.

Rebased on 6ab81d9. Node 20 CI fails on the archive regression; the same failure reproduces on exact main.

The full browser CI run caught a test that changed local storage after App initialized. Updated it in 876d252 to choose Codex through the UI, keeping the project-plus assertion intact. The prior run passed the other 621 tests; CI is rerunning.

Comment thread packages/harness/web/src/components/NewSessionComposer.tsx Outdated
@BYWallace
BYWallace force-pushed the brettwallace/sap-3121-template-harness branch from bba398d to f608e54 Compare September 8, 2026 17:57
Comment thread packages/harness/web/src/App.tsx Outdated
ratataque
ratataque previously approved these changes Sep 8, 2026

@ratataque ratataque left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a quick comment above, but otherwise it is much more cleaner now, thank you !

@BYWallace
BYWallace force-pushed the brettwallace/sap-3121-template-harness branch from 647c389 to ab0d99f Compare September 8, 2026 20:49
@BYWallace
BYWallace changed the base branch from main to codex/studio-archive-before-retention September 8, 2026 20:50
@BYWallace
BYWallace changed the base branch from codex/studio-archive-before-retention to main September 8, 2026 20:50
@BYWallace
BYWallace requested a review from ratataque September 8, 2026 21:10
@ratataque

Copy link
Copy Markdown
Contributor

The coding agents is good.

but the backfill stops after 200 conversations, but cleanup still runs afterward. If more than 200 need archiving, cleanup can delete the remaining conversations' raw events before they're archived, e.g. when they're older than 30 days. We should finish archiving the remaining batches before cleanup, and keep the source if archiving fails.

@BYWallace
BYWallace force-pushed the brettwallace/sap-3121-template-harness branch from d5169fc to 9dface7 Compare September 9, 2026 06:01
@BYWallace
BYWallace force-pushed the brettwallace/sap-3121-template-harness branch from 9dface7 to 1f3869a Compare September 9, 2026 06:29
@BYWallace

Copy link
Copy Markdown
Contributor Author

@ratataque thanks again, kept the 200-record limit to control startup writes. Cleanup now waits until archiving is complete. If a pass reaches the limit or fails, we keep the source events and continue on the next scheduled pass.

added tests for 200+ conversations, failures, and archive eviction between passes as well

@ratataque ratataque left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you ! LGTM 👍

@BYWallace
BYWallace merged commit 7776065 into main Sep 9, 2026
11 checks passed
@BYWallace
BYWallace deleted the brettwallace/sap-3121-template-harness branch September 9, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants